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SUMMARY 


The outside scene is often an important source of information for manual 
control tasks. Important examples of these are car driving and aircraft 
control. This paper deals with modelling this visual scene perception 
process on the basis of linear perspective geometry and the relative motion 
cues. 


Model predictions utilizing psychophysical threshold data from base-line 
experiments and literature of a variety of visual approach tasks are 
compared with experimental data. Both the performance and workload results 
illustrate that the model provides a meaningful description of the outside 
world perception process, with a useful predictive capability. 


INTRODUCTION 


Many manual control tasks depend on the visual perception of the 
outside scene. In the context of aircraft control, the most important 
example is the visual approach scene. So, in order to investigate a great 
many flight situations in the approach and landing, it is mandatory to take 
into account this visual scene perception process which has often a major 
impact on mission performance. 


Based on a concise inventory of the most important characteristics 
(cues) of the visual scene the visual scene perception process is described 
(modelled) on the basis of the linear perspective geometry and the relative 
motion cues. This involves mathematical relationships between these visual 
cues and the aircraft state variables. After linearization this model can be 
integrated in the exisiing framework describing piloted aircraft behavior 
(the optimal control model). This is the subject of the next chapter. 

The visual scene perception model involves assumptions concerning 
perceptual thresholds of the various cues, noise levels associated with 
observing these cues and interference among them. Values for these parameters 
are derived from baseline experimental data supplemented by the psycho- 
physical literature. Based on these values a theoretical analysis is 
performed dealing with a variety of visual approach conditions. 
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Furthermore, the results of an experimental program are compared with the 
model predictions. In addition, model predictions of pilot workload are compared 
with subjective ratings. 


VISUAL SCENE PERCEPTION MODEL 


One of the earliest studies of visual scene perception directly related 
to flight control problems has been performed by Gibson (Refs. 1 and 2). 
According to Gibson, the most important visual cues which can be derived 
from the visual field are related to 

. the linear perspective geometry 
. relative motion or motion parallax 
. the apparent size of objects whose real size is known 
. a far object covered by a near one 
. the distribution of flight and shade over an object 
. aerial perspective and the loss of detail with distance. 

Of these, the linear perspective geometry provides a variety of cues. 
This is illustrated by the schematic version of the visual scene in figure la 
which can be thought of to consist of lines and points (textural elements). 
This involves not only the linear and angular position of the observer with 
respect to the outside world but also (dynamically) the relative motion. 

The point of the visual field toward which the observer is moving appears to 
be stationary ("focus of expansion"). All other textural points move with 
respect to the observer which can be indicated by velocity vectors 
("streamers"). This is shown in figure 1b for the case of rectilinear motion. 
Various other references mention visual cues which can be conceived as 
examples of the afore-mentioned basic elements. Most of them are related to 
the landing approach scene (Refs. 3-5). 

From the foregoing it can be derived that a reasonable approach is to 
model the visual scene perception process on the basis of the linear 
perspective geometry and the relative motion cues. Following reference 6 
this involves a description of the cues which can be derived from the visual 
scene and their functional relationships with linear and angular positions 
and velocities of the observer. When, in addition, the relationships between 
the moving observer and the visual scene can be linearized about a nominal 
condition, the perception process can be described in standard estimation 
theoretical terms and included in the optimal control model structure in the 
following manner. 

Let the observer (aircraft ) moving with respect to the outside world 
be described by the system state x(t). This involves the common linear and 
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Although the following applies to a variety of man-machine situations, 
this analysis is directed at the aircraft control problem. 
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angular positions of the aircraft as veil as additional parameters to describe 
reJevant characteristics of the moving visual scene (with respect to the 
aircraft). After linearization about a nominal path the result will be a set 
of linear, tin general) time-varying equations given by 

x(t) = A(t) x(t) + 3(t) w(t) (1) 


where A(t) describes the process of the aircraft moving with respect to the 
outside world, and w(t) represents system disturbances (e.g. turbulence). 
Furthermore, the visual cues will be described by the display vector y(t). 
The relationships between these displayed variables and the system state is 
given by 


y(t) = C(t) x(t) 


( 2 ) 


The perception of these variables is accompanied with an equivalent time 
delay, perceptual thresholds and observation noises. Also the interference 
between the various visual cues, a.o. arizing from the necessity to scan the 
visual field and to divide the attention among the various cues, has to be 
considered (Refs. 7 and 8). Wow, these observations of the visual scene are 
dealt with in the same fashion as observations from other sources {e.g. 
displays, motion cues, etc.). The system state is estimated optimally (by 
means of a Kalman-Bucy filter) on the basis of the known (learned) dynamics 
involved and the observations. This state estimation process can be considered 
as an internal representation of the task environment. 


Relationships between visual scene characteristics and the system state 

A schematic version of the visual scene (Fig. 1) can be assumed to 
comprise textural elements and known objects. Both provide linear perspective 
geometrical cues (basically, the inclination of lines) and impressions of 
relative position and velocity. 

The inclination ft , of a line element of the visual scene is given by 

ft = tan" 1 Y/H (3) 

where Y is the distance betwe n the observer and the pertinent line element 

perpendicular to the looking direction and H is the vertical position of the 

observe..'. Assuming small perturbations (y, h and co) around the trim condition 

(Y , H and ft ) results after some manipulation (to a first order) in the 

'.o*oo. 

linear expression 

co = C, h + C y (Ha) 

h y 

where 

C v = -sin 2 ft /2H (Hb) 

h oo 

Differentiating eq (Ha) yields the expression for the inclination rate 
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cu = Cft + C y (4c) 

n y 

The small perturbations of the relative position and velocity of an 
element of the visual scene is simply given by 

a h = h/R ; a y = y/R 

and ^ ^ ^ 

“h = * “y = 

where a is the visual angle and R is the distance between the object and 
the observer. 

Furthermore, when the attitude of the observer (aircraft) is taken into 
account (with attitude angles cp, 0 and <jt) eqs. (4) and (5) become 


ft) = C, h + C y + cp 
h y 

a, = h/R + 0 
h 

a = y/R + <l> 

v 

and the corresponding time-derivatives a> = ...» etc. 


( 6 ) 


Next, these expressions are utilized to describe the cues which can be 
derived from the visual approach scene. 


Visual approach scene 

A schematic version of the visual approach scene is shown in figure 2. 

The cues which are assumed to be derived from this scene are indicated. 

The most important cue for lateral guidance is derived from the inclination 
of the runway sides and/or centerline. The lateral deviation y, is zero if 
the inclination of both runway sides is the same (to^ = cu^) and the inclination 
of the centerline is zero (co = 0). 

Vertical guidance has to be Based on the (average) inclination of the runway 
sides when no runway end and no horizon is visible. In that case, the 
observer has to know the nominal inclination {Q ), which is range-varying. 

The following model analysis and experimental results will show that a better 
indication of the vertical position is obtained when the length of the 
runway a (or, almost equivalently, the depression of runway threshold with 
respect to the horizon) is visible. Also in that case, the observer has to 
know the nominal depression which is, however, constant during a standard 
approach (e.g., 3 deg). 

Glide slope information requires also the estimation of the distance to 
touchdown. This can be based on the apparent size of ground objects, of which 
the most important is often the runway width. 

Aircraft attitudes provide "inner-loop" information and can be derived from 
the relative position and inclination of (e.g.) the horizon and any aircraft 
reference. The pitch angle 0, which has to be estimated with respect to its 
(non-zero) nominal value and the bank angle cp are indicated in figure 2. 
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MODEL ANALYSIS 
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The linear visual scene perception model (VSPM) can be implemented xn 
the optimal control model (Refs. 6 and 7). Based on the foregoing discussion, 
a variety of visual approach conditions are selected to analyze theoretically. 
In addition, an experimental program has been conducted to provide a critical 
test for the hypotheses (assumptions) underlying the model results. In order 
to obtain detailed information concerning the information processing involved 
in the manual approach task, no range-varying effects are considered in the 
following analysis. In other words, it is assumed that the aircraft is "frozen" 
at a fixed point of the approach path corresponding with a nominal altitude of 
200 ft for a 3 approach ("hovering"). The consequence is a stationary process 
involved allowing frequency domain measures such as human describing functions 
and observation noise spectra. Especially the latter will provide a sensitive 
check on the exactness of the values used for the model parameters under 
investigation. The primary model parameters are the perceptual thresholds of 
the various visual cues (display elements) involved because these represent 
the most uncertain model parameters. The results of several previous 
experimental studies suggest reasonable accurate values for the remaining model 
parameters . 

Therefore, base-line experiments have been conducted and relevant psycho- 
physical literature have been searched resulting in reasonable reliable 
estimates for the perceptual thresholds involved. Finally, the last section 
contains the model analysis proper and the resulting model predictions. 


Visual scene configurations 


Referring to the foregoing discussion the configurations given in figure 
3 were selected for the following model analysis and formal experiment. 

Vertical control on the basis of the inclination of the runway sides 
can be compared with the condition that the depression of the runway 
threshold below the runway end (a) or below the horizon is visible 
(configurations 1 and 2). Furthermore, the effect of an aircraft reference 
providing explicitly pitch information is of interest (configuration 3). 

Lateral control utilizing the inclination of the centerline is 
represented by configuration h. In case the runway sides are available, the 
inclination of both sides has to be estimated and compared with each other 
(configuration 5). A simple model analysis shows that this process is 
associated with the same observation noise as in the case of a center line. 
Only the perceptual thresholds involved are different (next section). This 
will be tested against the experimental results. Again the effect of 
explicit roll information provided by the aircraft reference is considered 
by including configuration 6. Configuration 7 concerns roll tracking based 
on the aircraft reference. This (presumably) easy task is included to evoke 
some variation in workload in order to yield additional experimental 
evidence for the workload model of reference 8 and to test the perceptual 
threshold assumptions involved. 
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Configurations 8 and 9 are selected to investigate the interference 
between vertical and lateral control. It is assumed that the pilot has to 
divide his attention between the various display variables (visual scene 
cues) involved. This interference is assumed not only within a control task 
(e.g. attention has to be divided between pitch angle and altitude) but also 
between vertical and lateral control when performing both tasks simultaneously. 
This represents a crucial hypothesis which will be tested in the following 
as the visual scene is widely assumed to represent integrated information 
and it is a non-trivial question whether the visual scene can be "broken 
down" into separate elements. Finally, configuration 10 is included to 
investigate the effect of additional texture. This has, in principle, its 
implications for the information contents of the visual scene which turned 
out to be of no interest but also for the psychological aspects (perspective 
illusion and realism) . 


Perceptual thresholds 

It was anticipated that perceptual threshold phenomena could be 
important for the foregoing visual scene cues. Thresholds can be accounted 
for in the optimal control model by modifying the observation noise 
covariance associated with a particular visual cue. 

Although the psychophysical literature reports a wealth of emperical 
threshold data, these data are known to be affected by numerous experimental 
conditions which easily explains the typical scatter in "comparable" data. 
Therefore, a baseline experiment has been conducted to determine the position 
thresholds of the display elements involved in the visual scene configurations 
shown in figure 3. These thresholds are primarily due to the lack of explicit 
visual references concerning zero or nominal, visual scene conditions. 

This involves that learning (experience) and temporal cues (memory 
functioning) are important in measuring and interpreting thresholds. 

Experimental details are given in reference 9« The resulting measurements 
are "translated" to values suitable for (as required by) the describing 
function representation for the assumed dead-zone non-linearity. The results 
are summarized in table 1 . 

As discussed in reference 9 thresholds associated with the perception 
of motion in the visual field can be related to resolution properties. This 
implies that the motion detection thresholds can be inferred from the fore- 
going discrimination data. The result is also contained in table 1 , 

Apart from these (nominal) threshold estimates, in table 1 it is also 
indicated how reliable these estimates are assumed to be. A sensitivity 
analysis in the following will serve to relate this incertainty in threshold 
values to a confidence interval associated with the system performance 
predictions of the model. 


Model predictions 


A block diagram of the control task(s) is given in figure h. System 
disturbance enters the system parallel to the control input. The resulting 
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output is displayed to the human operator as the pitch and roll angle (for 
the pertinent configurations) representing K-dynamics. The integral of these 
outputs are the altitude (or approach angle) and lateral deviation (or center 
line inclination), respectively (K/s-dynamics) . The disturbances are white 
noise processed by two first order filters with poles at one rad/sec and two 
rad/sec. The disturbance levels are f^r the vertical task given by a 
resulting pitch variance of 0.068 d^g and for the lateral task given by a 
resulting roll variance of 10.5 deg (corresponding with the values used for 
the experimental program) . Details concerning sensitivities and gains 
involved are contained in reference 9» 

Model parameters can be divided in parameters which are constant for all 
configurations and parameters which are considered as the remaining model 
variables. Also the experimental results of the next chapter will be related 
to these (dependent) variables. The key variables are the perceptual thresholds. 
The nominal values of table 1 are assumed for the model predictions. 

Furthermore, the effect of the upper- and lower threshold, values on the 
system outputs is also determined and discussed in the following chapter. The 
overall level of attention (P ) is also, to same extent, variable, although 
this value has been shown in previous studies to be relatively constant. A 
nominal value of -20 dB is assumed and the effect of ± 2 dB on the system 
outputs is considered. 

The constant model parameters are: a neuro-motor time constant of 0.1 sec, 
a perceptual time delay of 0.2 sec and a motor noise ratio of -30 dB. 

Now, assuming that the human operator divides his attention among the 
visual cues (position and velocity of all display elements) optimally, i.e., 
minimizing the given cost functional* (Ref. 8), system performance can be 
predicted for the various configurations. The results are given in table 

Vertical control is superior for the condition that the runway 
depression angle and the pitch angle can be observed (conf. 3). The contri- 
bution of the pitgh information amounts to a 20 % reduction of the approach 
angle variance {o of conf. 2). When the viewing condition is such that no 
horizon or runway end is visible and control has to be based on the runway 
sides (ro^ and/or w ) and runway threshold variation (a) the vertical approach 
performance is degraded substantially. This clearly demonstrates the 
contribution of the various visual cues involved. Furthermore, in the case of 
both vertical and lateral control, the vertical approach performance is 
predicted to deteriorate with 30 % to 50 % (due to the assumed interference 
between both tasks). The last column of table 1 contains the (optimal) 
fractions of attention dedicated to the various cues. 

The best lateral approach performance is obtained when the runway 
centerline inclination (a> ) cue is available (conf. 4). Lateral control 
utilizing the runway sides is substantially degraded (conf. 5) due to the 
larger perceptual threshold of this cue. The bank angle provides useful 


* According to the instructions given to the subjects in the experiment 
the system output is assumed to be minimized. In addition the control rate 
is weighed yielding the neuro-motor time constant of 0.1 sec. 
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inner loop information (conf. 6). When performing the vertical and lateral 
task simultaneously, the model predicts a deterioration in lateral performance 
(confs. 8 and 9) of about 100 %. The model predicts that the effect of the 
texture (conf. 10) on system performance is negligible. 

The effect of the model parameter variations (thresholds and overall 
attention) on the system scores and additional theoretical results will be 
discussed in the next chapter where the model predictions will be compared 
with the experimental results. 


EXPERIMENTS 


The first objective of the experimental program was to test the fore- 
going model results with respect to both the fundamental hypotheses involved 
(optimality in control and attention allocation, interference between cues) 
and the assumed numerical values of the key model parameters. Secondly, in 
case significant discrepancy occurs between model and experimental results 
the appropriate adjustments can (hopefully) be made in the model assumptions 
underlying the model results. 


Experimental procedures 

The same 10 configurations as discussed previously are investigated in 
the experimental program. These configurations were four times presented to 
the (four) subjects (general aviation pilots) in a random order. Each run 
lasted 200 sec. Between the runs the subjects were asked to give their 
impression of the exerted workload (Reference 9 contains the rating scales 
used and additional experimental details). The subjects were instructed to 
minimize the mean-squared system output. They were trained on the ten 
configurations in a random order till a relatively stable performance level 
was reached. All together, about 250 training trials were performed. 

An analog computer was used to simulate the vehicle dynamics and to 
generate the visual scene characteristics. This visual scene was presented to 
the subjects on a TV monitor located 2.5 m in front of their point of regard. 
They manipulated a two-axis isometric hand control. The system parameters 
were recorded on FM magnetic tape for off-line mean-squared scores and 
frequency domain computations*. 

Comparison of experimental results and model scores 

In this section the experimental results in terms of mean-squared 
performance scores are compared with the model predictions. Based on the 
results of table 2 firstly the approach angle (a)- and centerline inclination 
(to ) scores are considered (the model predicts attitude- and control scores 
which are relatively insensitive over the configurations). 


X * 

Unfortunately, these frequency domain data were not available in time to 
include in this paper. These results will be included in reference 9. 
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Apart from the nominal model predictions (of table 2) the effect of the 
uncertainty in underlying assumptions (i.e. , numerical values of the thres- 
holds a..l overall attention) on system performance is determined. For the 
upper- and lover threshold values given in table 1 and, in addition, + 2 dB 
and -2 dB variation in overall attention the corresponding performance scores 
are determined. It is hypothesized that the experimental scores lie within 
the resulting performance interval. 

In figures 5 and 6 both the experimental means and standard deviations 
(of 16 runs) and the model predictions are given. For all single-axis tasks 
the experimental scores lie well within the predicted interval. This indicates 
not only that the model is "right" but also that the assumed numerical 
values for the thresholds and overall attention are close to the "real" 
values . 

For the dual-axis tasks the experimental results do clearly not match 
the model predictions. The experimental data of configuration 9 and 10 have 
been pooled because both the model predictions and the experimental results 
for both configurations indicate that the only effect of the texture 
information is the enhancement of the perspective illusion. This was also 
apparent during the learning phase. An adjustment of the model parameter 
values (which has to be appropriate for the single-axis tasks as well) does 
not result in a good agreement with the experimental scores. Therefore, it 
is tentatively concluded that the assumed hypothesis of interference between 
the two tasks has to be rejected. Instead, the following hypothesis is 
considered: the visual scene stimulates the human operator to perform the 
dual-axis task just as well as the single-axis task (thus, vertical control 
is not degraded when the lateral control task is added, and vice versa). So, 
it is assumed that there is no performance interference . This will be further 
discussed in the following. 

Comparing also the attitude scores (9 and cp) and the control scores 
(6 and b ) of the model predictions in table 1 and the measured scores 
given in table 3 it is apparent that both the measured attitude scores and 
the measured control scores are much lower than predicted. This indicates 
that the subjects (being pilots) performed the - to some extent realistic 
appearing - "approach" tasks in a much smoother fashion than the model 
predicts on the basis of an assumed neuromotor time constant of 0.1 sec. This 
is confirmed by pilot commentary indicating that the pilots were reluctant to 
make rapid control movements and "chase the needles". 

Based on this observation the neuromotor time constant was adjusted to a 
value of 0.25 sec. This value which was kept constant in the following 
analysis is apparently more representative for outer-loop control behavior. 

In addition, figure 5 suggests that for the vertical control tasks a better 
agreement between measured and model results will be obtained when the lower 
threshold values given in table 2 will be assumed (0.2 /sec and 0.4°). This 
is the only minor adjustment of the model variables. 

The resulting model scores are compared with the measured mean-squared 
values in table 3. In general, the agreement between the measurements and 
the refined model scores is quite good. Now (with a neuromotor lag of 0.25 
sec) the control scores match, on the average, very well. The same can be 
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said of the system outputs a and, to less extent, a> . 

A comparison of the pitch attitude scores shows tha£ the pilots were somewhat 
more conservative in making pitch corrections than the model predicts (apart 
from configuration l). These lower pitch scores (and the corresponding some- 
what lower control scores) could easily be duplicated by the model, however, 
by an appropriate weighting of the pitch angle. The mean-squared roll angles 
match again, rather well . 

The system output scores are summarized in figure 7. For the dual-axis 
configurations, both the scores corresponding with the assumption that there 
no performance interference between the two axes and the "full interference" 
scores are indicated. The results strongly support the hypothesis that there 
is no interference between the vertical- and lateral axis thanks to the 
visual scene. 

In summary, it can be concluded that for the relatively realistic, outer- 
loop control tasks under investigation a neuromotor time constant of (say) 

0.25 sec is appropriate. Furthermore, only one minor adjustment of the 
nominal model variables was required to yield, on the average, a good 
agreement between model results and measurements: .a position threshold for a 
and Q of 0.4° and a velocity threshold for a and 6 of 0.2 /sec (the same 
value as found in reference 10). Finally, the experimental results provided 
convincing support for the hypothesis that the visual scene perception 
process can be described on the basis of the, mutually interfering, various 
(separate) visual cues considered. There is no performance degradation 
(interference) when both the vertical and lateral control task are performed 
simultaneously . 


Workload model results and subjective ratings 

Using the foregoing model results human operator workload can be computed. 
The workload model (a.o. discussed in reference 8) involves not only the 
level of attention, P , dedicated to the task in accordance with the model 
of reference 11, but also the aspect of arousal ("uncertainty"). 

The model predictions are compared in figure 8 with subjective ratings 
on the workload scale given in references 8 and 9* Apart from configuration 
1 the linear correlation between subjective ratings and workload model 
predictions is quite good (r = 0.88). This result provides additional 
support for the workload model. 

The model predicts a much lower workload level for configuration 1 than 
reflected by the subjective ratings. The explanation for this is that for 
this configuration the subjects were not sure what the right (nominal) 
vertical position was. Not only they learned slowly on this configuration 
(somewhat discouraged by their varying learning-performance) but also they 
clearly did not like the uncertainty involve! in performing the task which 
can also be related to training. So, the model, not including this learning 
aspect, predicts that the workload corresponding with this configuration will 


* For the roll-only task (conf.7) an overall level of attention, P , of 
-18 dB had to be assumed in order to match the measured scores. 0 
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substantially reduce when the subjects are more trained on (familiar with) 
this task. 


CONCLUSIONS 


The visual scene provides a variety of perspective geometrical and 
relative motion cues. The experimental results have supported that these 
characteristics can be considered as separate cues among which the human 
operator has to divide his attention. The commonly accepted idea that 
pictorial information is better integrated (less interfering) than separate 
display elements is in the present study specifically demonstrated in that 
there is no performance interference between the vertical and lateral task. 
Both the workload model results and the subjective ratings indicate that 
the workload is increased indeed when performing both tasks. 

In the case of guidance control tasks (e.g., the visual approach task) 
pilots are reluctant to make rapid control movements. This is represented 
in the optimal control model by a weighting on control rate corresponding 
with a neuromotor time constant of about 0.25 sec. This outer-loop control 
behavior is distinguished from attitude (inner-loop) control tasks which 
can be modelled with a neuromotor time constant of 0.1 sec (Ref. 7). 

Furthermore, the assumptions concerning the key parameters of this 
investigation, i.e. the perceptual thresholds, could (indirectly) be 
checked against the experimental data. Apart from one minor adjustment the 
a priori assumed threshold values yielded a good agreement between model 
scores and measurements. The sensitivity analysis visualized in figures 5 
and 6 indicates that this result allows a reasonable accurate verification 
of the underlying model parameters (thresholds and level of attention). 

Finally, the workload model predictions have been confirmed convincingly 
by subjective ratings. Apart from configuration 1 (the performance of which 
task must have been dominated by a psychologigal effect not included in the 
model) the linear correlation between model predictions and subjective 
ratings was 0.9. 
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Table 2 Model predictions 
a) VERTICAL CONTROL 



b) LATERAL CONTROL 
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Table 3 Comparison of measured scores and model results 


VERTICAL 

CONFIGURATION 


9 » 10 


MS (deg; 

a 


measured 


0.065 


MS ft (deg ) 


0.01*0 


MS* (M ) 
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e 



LATERAL 

CONFIGURATION 


MS (deg*) 

to 

C 


MS ( deg ) 
<P 


MS ft (N ) 
























































































Ill RELATIVE MOTION OP THE VISUAL FIELO 


Fig. 1: Visual scene 
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Fig. 2: Cues derived from the visual scene 
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N.8. CONFIGURATION 7 CONCERNS ROLL TRACKING BASED ON ROLL BAR ONLY 

Fig. 5: Viewing conditions and selected configurations 
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Fig. H: Control task 



Fig. 5: Comparison of experimental scores and model 

predictions - vertical control 
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CONFIGURATIONS 


Fig. 6: Comparison of experimental scores and model 

predictions - lateral control 
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CONFIGURATIONS 


Fig. 7: Refined model scores and experimental results 
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Fig. 8: Subjective workload ratings and 
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